Some electronic devices, such as smartphones and tablet computers, include or are associated with speech recognition capabilities that enable users to access functionality of the device via speech input. Speech input is processed by an automatic speech recognition (ASR) system, which converts the input audio to recognized text. Electronic devices may also include or be associated with a natural language understanding (NLU) engine that interprets user input and takes an action based upon determined semantic content of the user's input (e.g., by facilitating actions with one or more applications accessible via the electronic device). Virtual agents or virtual assistants are one such class of applications that benefit from NLU processing to assist users in performing functions such as searching for content on a network (e.g., the Internet) and interfacing with other applications. Users can interact with a virtual agent by typing, touch, speech, or some other interface. To determine a meaning of a user input, the NLU engine interprets the user input, and a virtual agent may attempt to infer an action the user wants to perform based on the NLU result.